Virtual reality system and method

ABSTRACT

A method comprises displaying in virtual reality a computer-generated scene; obtaining a movement command from a real-world physical movement of a user, the movement command corresponding to a movement of a virtual body; and adjusting the movement of the virtual body in dependence on an effect of gravity in the computer-generated scene and/or in dependence on the presence of at least one object within the computer-generated scene that inhibits the movement of the virtual body, wherein the adjusting of the movement is such that the adjusted movement of the virtual body does not correspond with the real-world physical movement of the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/357,293, filed Jun. 24, 2021, entitled Virtual Reality SystemObtaining Movement Command from Real-World Physical User, which is acontinuation of U.S. patent application Ser. No. 15/784,718, filed Oct.16, 2017, entitled: Virtual Reality System Obtaining Movement Commandfrom Real-World Physical User, which claims the benefit of UK PatentApplication No. 1617446.8 filed Oct. 14, 2016 and U.S. ProvisionalPatent Application No. 62/410,107, filed Oct. 19, 2016, which areentirely incorporated herein by reference.

FIELD

The present invention relates to virtual reality, for example to asystem and method for viewing architectural structures in virtualreality.

BACKGROUND

Virtual reality may refer to a method of displaying a computer-generatedthree-dimensional environment such that a user perceives that they areinteracting with that environment, for example walking through theenvironment. In some known virtual reality systems, a user wears avirtual reality headset (which may be called a head-mounted display).The virtual reality headset presents images on screens in front of theuser's eyes. The images may create an impression of an immersivethree-dimensional environment.

The user may interact with the computer-generated environment by, forexample, moving his or her head, hands, or other body parts; orproviding input via a hand-held device such as a games controller, wand,keyboard, or mouse.

The computer-generated environment may be representative of a realenvironment. For example, the computer-generated environment may berepresentative of a real building, street or natural environment.

In some circumstances, virtual reality may be used for architecturalvisualisation. The computer-generated environment may be representativeof a structure (for example a building) that has not yet beenconstructed, or a structure that is being modified. A user may view acomputer-generated image of the structure as if positioned inside thestructure, for example as if moving from room to room within thestructure or looking around a room of the structure.

SUMMARY

In a first aspect of the invention, there is provided a methodcomprising displaying in virtual reality a computer-generated scenerepresentative of a structure; receiving a user input representative ofa variation of a time or weather parameter; and updating thecomputer-generated scene in dependence on the variation of the time orweather parameter, thereby enabling the user to vary on command anappearance of at least part of the structure in the computer-generatedscene to represent a change in time and/or weather conditions.

The computer-generated scene may be updated to show a different time, ordifferent weather, from that initially represented in thecomputer-generated scene. Using the method, a user may view how astructure, for example a building, may look at different times or indifferent weathers. The user may comment on features of the structureand/or make changes in the structure. For example, the user may be anarchitectural client. By seeing a virtual representation of thestructure as if viewed at different times and/or in different weatherconditions, the user may approve or disapprove features of thestructure, or suggest changes to the structure.

The updating of the computer-generated scene may comprise live updatingof the computer-generated scene while the computer-generated scene isviewed in virtual reality by the user.

The user may vary time or weather while viewing the computer-generatedscene. The variation of time or weather may be made in combination withother changes. For example, the user may vary the time or weather whilemoving around the computer-generated scene. The user may vary the timeor weather while also varying a viewing position from which the user isviewing the computer-generated scene.

The user input representative of the variation of the time or weatherparameter may be provided by the user via a handheld input device. Thehandheld input device may comprise at least one of a games controller, awand.

The user input representative of the variation in the time or weatherparameter may be provided by operating a user input tool. The user inputtool may comprise a button, and the user input may comprise holding downthe button. The user input tool may comprise a slider, and the userinput may comprise sliding the slider. The user input tool may comprisea pressure-sensitive device, and the user input may comprise pressingthe pressure-sensitive device. The user input tool may be real orvirtual.

The user input representative of the variation in the time or weatherparameter may be provided via the handheld input device by holding downa button on the handheld input device.

The handheld input device may provide a simple input mechanism that maybe easy for the user to control. The handheld input device may be easyto control even when the user is simultaneously performing otheractions, for example walking around the computer-generated scene. Forexample, the user may not have to select or type numerals in order toinput a time. The handheld input device may be familiar and/or intuitiveto the user.

The variation in the time or weather parameter may comprise an incrementin time. The method may further comprise determining the increment intime in dependence on a length of time for which a or the user inputtool is operated. The method may comprise determining the increment intime in dependence on a length of time for which a or the button is helddown by the user.

The user may change time by using a single button control. Control via asingle button may be simple for the user.

The computer-generated scene may be updated to represent a changed timewithout representing intermediate times between an initial time and thechanged time. Updating without representing intermediate times may befast and may provide the user with the information they have requestedwithout providing other times which may be irrelevant to the user.

The computer-generated scene may be updated to represent intermediatetimes between an initial time and the changed time. Representingintermediate times may give the user an impression of time progressing.Representing intermediate times may provide a change that appears to becontinuous. Representing intermediate times may provide a presentationthat resembles a time-lapse video showing changes over time (forexample, the effect of a succession of different sun positions).

The variation of the time or weather parameter comprises a variation inat least one of a time of day, a time of year, a type of weather.

The user may view the structure as if viewed at different times of day,at different times of year, and/or in different types of weather.

The method may comprise determining a sun position in dependence on atime of day and on a geographical location. The displaying and/orupdating of the computer-generated scene may comprise generating thescene using the determined sun position.

The determining of the sun position may be further in dependence on a orthe time of year.

The appearance of the structure at different sun positions may beparticularly important to the user. For example, the user may be anarchitectural client. The user may want to see how much light isavailable in a particular room of the structure at a particular time ofday, for example to see whether the lighting is inadequate or whetherthe lighting is excessive (for example, causing glare).

The at least one weather parameter may be determined based on at leastone of the time of day, the time of year.

The variation of the weather parameter may comprise a variation of atleast one of an extent of cloud coverage, a type of precipitation, arate of precipitation, an amount of fog, an amount of smog.

The computer-generated scene may be representative of an interior of thestructure. The variation of the time or weather parameter may comprise avariation of a sun position and/or type of weather visible from theinterior of the structure via at least one door or window.

The method may further comprise displaying with the computer-generatedscene a time display. The method may further comprise displaying withthe computer-generated scene a calendar display. The method may furthercomprise displaying with the computer-generated scene a map display. Themethod may further comprise displaying with the computer-generated scenea weather display.

A time or calendar display may indicate to the user a time of day oryear that is currently represented in the computer-generated scene. Amap display may indicate to the user a geographical location of thestructure and/or of the user. The weather display may indicate to theuser a type of weather that is currently represented in thecomputer-generated scene.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device, and at least oneprocessor configured to: display in virtual reality on the at least onedisplay screen a computer-generated scene representative of a structure;receive from the at least one user input device a user inputrepresentative of a variation of a time or weather parameter; and updatethe computer-generated scene in dependence on the variation of the timeor weather parameter, thereby enabling the user to vary on command anappearance of at least part of the structure in the computer-generatedscene to represent a change in time and/or weather conditions.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: receiving photographic datarepresentative of a geographical area, wherein a second part of thegeographical area is further from a structure of interest than a firstpart of the geographical area; processing the photographic data toobtain a three-dimensional representation of the first part of thegeographical area and a two-dimensional representation of the secondpart of the geographical area; and displaying in virtual reality acomputer-generated scene representative of the structure of interest,wherein the computer-generated scene comprises the three-dimensionalrepresentation of the first part of the geographical area and thetwo-dimensional representation of the second part of the geographicalarea.

The three-dimensional representation and two-dimensional representationmay be obtained by processing the same photographic data.

The first part of the geographical area may comprise or be adjacent tothe structure of interest.

By using photographic data, a realistic representation of thegeographical area may be obtained. For example, the photographic datamay comprise a plurality of realistic images.

The three-dimensional representation and two-dimensional representationmay be obtained from the same photographic data or from closely matchingphotographic data, for example photographic data that is matched in timeof day and/or camera settings. The three-dimensional representation andtwo-dimensional representation may be a close visual match to eachother. Structures, for example buildings, in the three- andtwo-dimensional representations may have similar appearance in thecomputer-generated scene. For example, the three-dimensional andtwo-dimensional representations may have similar tone, lighting,resolution, hue, brightness, colour parameters, or other imagequalities.

By using a two-dimensional representation of a part of the geographicalarea that is further from the structure of interest, in somecircumstances better performance may be achieved than may be obtained ifa three-dimensional representation of the whole of the geographic regionwere to be used. For example, an amount of processing power used may bereduced and/or a speed of image generation may be increased.

By using a combination of a three-dimensional representation for a partof the geographical area that is nearer to the structure of interest, abetter representation of structures near to the structure of interestmay be obtained than if a two-dimensional representation were to beused. For example, a position of nearby structures may be represented inan accurate manner.

The photographic data may comprise at least one of aerial data, dronedata.

Aerial data (which may for example be drone data) may be used to createa representative view outside the structure of interest, for example arepresentative view outside a building. The use of drone data may allowup-to-date photographic data to be acquired at a moderate cost. Forexample, if the structure of interest is a building that has not yetbeen built or that is to be modified, photographic data of ageographical area around a site or proposed site of the building may beobtained specifically for the purpose of displaying the building invirtual reality. Alternatively aerial data may be used that haspreviously been acquired for another purpose, for example data that hasbeen acquired for mapping, planning or surveying.

The two-dimensional representation of the second part of thegeographical area may comprise at least one static two-dimensional imageof the second part of the geographical area.

The computer-generated scene may be displayed as if viewed from aviewpoint within or near the structure of interest. The or each statictwo-dimensional image may be a two-dimensional image of at least part ofthe second part of the geographical area as viewed from a viewingposition within or near the structure of interest, which may or may notbe the same viewing position from which the computer-generated scene isdisplayed. In some circumstances, displaying the computer-generatedscene from a different viewing position than the viewing position fromwhich the static two-dimensional image or images are viewed may notcause significant inaccuracy in the view presented. The second part ofthe geographical area may be considered to include structures that arein the far distance. A small difference in viewing position may not makemuch difference to the positions of structures in the far distance asviewed from the structure of interest.

The two-dimensional representation of the second part of thegeographical area may comprise a single panoramic image.

The single panoramic image may be easily stored and/or easily used. Thesingle panoramic image may provide a convenient representation ofstructures further from the structure of interest, for examplestructures in the far distance. The single panoramic image may provide aphotographic backdrop.

The three-dimensional representation of the first part of thegeographical area may comprise a plurality of three-dimensional objects,wherein each of the plurality of three-dimensional objectsrepresentative of a respective structure in the first part of thegeographical area.

By representing structures in the first part of the geographical area asthree-dimensional objects, the apparent position of the structures inthe first part of the geographical area may be made to change independence on viewing position. A relative position of structures in thefirst part of the geographical area may change with viewing position.

Displaying in virtual reality the computer-generated scene may comprisedisplaying the computer-generated scene as if viewed from a firstviewing position and subsequently displaying the computer-generatedscene as if viewed from a second viewing position. An appearance of thetwo-dimensional representation may be substantially unchanged betweenthe first viewing position and the second viewing position. Anappearance of the three-dimensional representation may be different whenviewed from the second viewing position than when viewed from the firstviewing position.

Changing the position from which the computer-generated scene is viewedfrom the first viewing position to the second viewing position maychange an apparent position of at least one structure in thethree-dimensional representation relative to structures in thetwo-dimensional representation. Changing the position from which thecomputer-generated scene is viewed from the first viewing position tothe second viewing position may not change an apparent position of astructure in the two-dimensional representation relative to otherstructures in the two-dimensional representation. Changing the positionfrom which the computer-generated scene is viewed from the first viewingposition to the second viewing position may change an apparent positionof a structure in the three-dimensional representation relative to otherstructures in the three-dimensional representation. Effects of parallaxmay be simulated. Providing apparent relative movement may provide amore accurate representation of structures that are near to thestructure of interest.

The method may further comprising determining the first part of thegeographical area and the second part of the geographical area.

The first part and the second part of the geographical area may bepre-defined. The first part and the second part of the geographical areamay be determined manually. The first part and the second part of thegeographical area may be determined automatically.

The determining of the first part of the geographical area and thesecond part of the geographical area may be based on distance from thestructure of interest. The determining of the first part of thegeographical area may comprise determining a part of the geographicalarea that is within a threshold distance from the structure of interest.The determining of the second part of the geographical area may comprisedetermining a part of the geographical area is above a thresholddistance from the structure of interest.

The threshold distance may be defined manually. The threshold distancemay be defined automatically. The threshold distance may be variable bya user. The photographic data may be re-processed using a differentthreshold distance. An updated computer-generated scene may be obtainedusing the re-processed photographic data.

The determining of the first part of the geographical area and thesecond part of the geographical area may comprise dividing a set ofstructures in the geographical area between the first part of thegeographical area and the second part of the geographical area.

The dividing of the set of structures may comprise respectivelyallocating each structure of the set of structures to the first part orto the second part of the geographical area based on at least one of: adistance of the structure from the structure of interest, a size of thestructure, an importance of the structure.

The first part of the geographical area is generally closer to thestructure of interest than the second part of the geographical area.However, in some circumstances, there may be an individual structure inthe second part of the geographical area that is closer to the structureof interest than an individual structure in the first part of thegeographical area. Structures may be allocated based on a distance fromthe structure of interest and on a size of the structure and/or animportance of the structure. For example, large and/or importantstructures at a given distance from the structure may be included in thethree-dimensional representation when smaller and/or less importantstructures at that distance. Certain structures may be prioritised forthree-dimensional representation, for example landmark structures.

The dividing of the set of structures in the geographical area betweenthe first part of the geographical area and the second part of thegeographical area may comprises allocating to the first part of thegeographical area structures at a distance to the structure of interestthat is below a threshold distance, and allocating to the second part ofthe geographical area structures at a distance to the structure ofinterest that is above the threshold distance.

The processing of the photographic data may comprise processing thephotographic data using at least one of map data, planning data,distance data, laser data.

The processing of the photographic data to obtain the three-dimensionalrepresentation may comprise processing the photographic data usingphotogrammetry.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device, and at least oneprocessor configured to: receive photographic data representative of ageographical area, wherein a second part of the geographical area isfurther from a structure of interest than a first part of thegeographical area; process the photographic data to obtain athree-dimensional representation of the first part of the geographicalarea and a two-dimensional representation of the second part of thegeographical area; and display in virtual reality on the at least onedisplay screen a computer-generated scene representative of thestructure of interest, wherein the computer-generated scene comprisesthe three-dimensional representation of the first part of thegeographical area and the two-dimensional representation of the secondpart of the geographical area.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: displaying in virtual reality acomputer-generated scene representative of a structure of interest,wherein the computer-generated scene comprises: a three-dimensionalrepresentation of a first part of a geographical area, and atwo-dimensional representation of a second part of the geographicalregion, the second part of the geographical area being further from thestructure of interest than the first part of the geographical area;wherein the three-dimensional representation of the first part of thegeographical region and the two-dimensional representation of the secondpart of the geographical region are obtained from the same photographicdata.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device, and at least oneprocessor configured to: display in virtual reality on the at least onedisplay screen a computer-generated scene representative of a structureof interest, wherein the computer-generated scene comprises: athree-dimensional representation of a first part of a geographical area,and a two-dimensional representation of a second part of thegeographical region, the second part of the geographical area beingfurther from the structure of interest than the first part of thegeographical area.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: displaying in virtual reality aninitial computer-generated scene comprising a miniature model of astructure at a first, smaller scale; receiving from the user a selectionof a location in the miniature model, the location in the miniaturemodel corresponding to a location in the structure; and displaying invirtual reality a further computer-generated scene representative of atleast part of the structure at a second, larger scale, wherein thefurther computer-generated scene is displayed as if viewed from aviewing position in the structure that is selected in dependence on theselected location in the miniature model.

Displaying a representation of a miniature model may provide an easyand/or intuitive method for a user to navigate the structure. The usermay directly interact with the miniature model. The user may select, forexample a particular room within the structure. The location in thestructure may be a viewing position that is of interest to the user.Selecting a location on the miniature model may be more straightforwardthan navigating to the selected location using another method, forexample walking through the structure in virtual reality to reach theselected location.

The initial computer-generated scene may be representative of at leastpart of the structure. The miniature model may be displayed in theinitial computer-generated scene as if the miniature model were presentin the at least part of the structure.

The initial computer-generated scene may be representative of adifferent part of the structure than is represented in the furthercomputer-generated scene. At least part of the furthercomputer-generated scene may be the same as at least part of the initialcomputer-generated scene.

Displaying the miniature model in the at least part of the structure mayallow the user to interact with the structure at two scalessimultaneously. The user may have the impression of being inside a partof the structure, while also looking at a scale model of the entirestructure which is provided by the miniature model.

The initial computer-generated scene may be displayed as if viewed froman initial viewing position. The initial computer-generated scene maycomprise an indicator indicating a location on the miniature model thatcorresponds to the initial viewing position.

By indicating a location from which the user appears to be viewing thescene, the user may understand their viewing position in context. Forexample, the user may understand which room of the structure they arecurrently viewing.

The displaying of the further computer-generated scene may comprisedisplaying a or the representation of the miniature model at a or thesmaller scale, wherein the representation of the miniature model isdisplayed in or adjacent to the further computer-generated scene.

Displaying the miniature model in both the initial scene and the furtherscene may allow the user to understand in context a position from whichthey are viewing the further scene.

The initial computer-generated scene may comprise an indicatorindicating the selected location in the miniature model. The furthercomputer-generated scene may comprise an indicator indicating theselected location in the miniature model.

The indicator may have a different visual appearance in the initialcomputer-generated scene than in the further computer-generated scene.For example, the indicator may have a first colour in the initialcomputer-generated scene (which is not viewed from a locationcorresponding to the selected location in the miniature model) than inthe further computer-generated scene (which is viewed from a locationcorresponding to the selected location in the miniature model).

In some embodiments, one indicator appearance (for example, one colour)is used for a location corresponding to a current viewing position, andanother indicator appearance (for example, another colour) is used for aselected location from which the scene is not being viewed.

The provision of indicators having different appearance may help theuser to distinguish where they are in the structure from where they haverequested to move to.

A transition from the initial computer-generated scene to the furthercomputer-generated scene may comprise teleporting the user to thelocation in the further computer-generated scene that corresponds to theselected location in the miniature model. Teleportation may provide aquick and simple way of moving around the structure.

The user may provide the selection of the location in the miniaturemodel by pointing an input device at the location in the miniaturemodel. The input device may comprise at least one of a handheld inputdevice, a games controller, a wand.

Pointing an input device at the miniature model may provide a quick andconvenient way of selecting a location to move to.

An apparent position of the miniature model may be the same in thefurther computer-generated scene as in the initial computer-generatedscene. The display of the miniature model may persist when a movement ismade. The miniature model may be permanently displayed.

The miniature model may be at least partially transparent. The miniaturemodel may be placed in a corner of a screen.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device, and at least oneprocessor configured to: display in virtual reality on the at least onedisplay screen an initial computer-generated scene comprising aminiature model of a structure at a first, smaller scale; receive from auser via the at least one user input device a selection of a location inthe miniature model, the location in the miniature model correspondingto a location in the structure; and display in virtual reality on the atleast one display screen a further computer-generated scenerepresentative of at least part of the structure at a second, largerscale, wherein the further computer-generated scene is displayed from aviewing position in the structure that is selected in dependence on theselected location in the miniature model.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: displaying in virtual reality acomputer-generated scene representative of a surface of interest and atleast one further surface; receiving from a user an annotation inputcomprising a first part and a second part, wherein at least the secondpart extends beyond the surface of interest; and drawing the annotationin the computer-generated scene such that the first part is drawn on thesurface of interest and the second part is drawn on an imaginary surfacedefined relative to the surface of interest.

The user may use an annotation that extends beyond one surface. By usingan annotation that extends beyond one surface, the user may indicate anextension of the surface of interest, or to highlight the surface ofinterest by drawing around it. It may be recognised that the user wishesto continue to draw in a plane of the surface of interest rather than tojump to another surface. The annotation may be more versatile than anannotation that is restricted to a single surface. The method ofproviding the annotation may be intuitive to a user, who may expect anannotation that starts in one plane (the plane of the surface) tocontinue in that plane, rather than jumping to the plane of a differentsurface.

The annotation input may be provided by the user using a virtualannotation tool.

The virtual annotation tool may be operated using an input device, forexample a handheld input device, controller, games controller, wand,mouse, or stylus. The annotation input may comprise an extended movementin space, for example a movement from left to right, right to left, upto down, or down to up. The annotation input may simulate drawing a lineor shape, for example a geometrical shape.

The virtual annotation tool may comprise at least one of a virtual laserpointer, a virtual pencil, a virtual pen, a virtual spray can.

The virtual annotation tool may be configured to project a virtualsubstance from a first viewing position.

The virtual substance may comprise at least one of light, paint, ink.The virtual substance may be projected from the first viewing positionso as to land on any surface in the line of sight of the virtualannotation tool.

The virtual annotation tool may provide a familiar effect, for examplean effect similar to that of a laser pointer, which projects light ontothe first object in its line of sight.

The method may further comprise selecting the surface of interest byprojecting the virtual substance onto said surface of interest.

The computer-generated scene may comprise a plurality of surfaces.Before the user starts to provide the annotation input, the surface ofinterest may not have been identified. The surface of interest may beidentified as the first surface that the user annotates using thevirtual annotation tool. It may be assumed that the user wishes tocontinue in the plane of the first surface, for example to draw anextension to the first surface.

The method may further comprise determining a transition between thefirst part and the second part of the annotation. The transition betweenthe first part and the second part may comprise a location at which thevirtual substance ceases to be projected onto the surface of interestand starts to be projected onto a or the further surface.

If the annotation input jumps from one surface to another (for example,as the beam of a laser pointer may appear to jump when moving from onesurface at a first distance from the laser pointer to another surface ata second distance from the laser pointer), the annotation input may beinterpreted (and drawn) as if it had stayed in the plane of the firstsurface.

The determining of the transition may comprise determining thetransition in dependence on a distance between the surface of interestand a further surface. The determining of the transition may be independence on a comparison between a threshold distance and the distancebetween the surface of interest and the further surface. If the distancebetween the surface of interest and the further surface is below thethreshold distance, no transition may be determined. The annotation maybe drawn on the surface of interest and on the further surface. If thedistance between the surface of interest and the further surface isabove the threshold distance, the part of the annotation in which theannotation tool projects onto the surface of interest may be drawn onthe surface of interest, while the part of the annotation in which theannotation tool projects onto the further surface may be drawn on theimaginary surface defined relative to the surface of interest.

The defining of the imaginary surface may comprise defining theimaginary surface relative to a normal of the surface of interest. Thenormal may comprise a normal at an initial point of annotation on thesurface of interest. The normal may comprise a normal at a final pointof annotation on the surface of interest. The normal may be calculatedas a vector perpendicular to a line connecting two previous annotationpoints.

The imaginary surface may be invisible in the computer-generated scene.

No representation of the plane may be rendered. The drawn annotation mayappear to extend into free space.

The imaginary surface may comprise an imaginary plane. The imaginaryplane may be aligned with at least part of the surface of interest.

The displaying of the computer-generated scene may comprise displayingthe computer-generated scene as if viewed from the or a first viewingposition.

The drawn annotation may appear to occupy the same position inthree-dimensional space when viewed from the second viewing position aswhen viewed from the first viewing position.

The method may further comprise displaying the computer-generated sceneand the drawn annotation from a second viewing position maintaining aposition of the drawn annotation in the computer-generated scene, suchthat an appearance of the drawn annotation from the first viewingposition is different from an appearance of the drawn annotation fromthe second viewing position.

The drawn annotation may maintain in the computer-generated scene aconstant position relative to the surface of interest and the at leastone further surface.

The method may further comprise displaying the computer-generated sceneand the drawn annotation from a second viewing position such that thedrawn annotation moves relative to the surface of interest and the atleast one further surface. The drawn annotation may appear to occupy adifferent position in three-dimensional space when viewed from thesecond viewing position than when viewed from the first viewingposition. The drawn annotation may rotate relative to the surface ofinterest and the at least one further surface to maintain a constantappearance relative to the viewer.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device, and at least oneprocessor configured to: display in virtual reality on the at least onedisplay screen a computer-generated scene representative of a surface ofinterest and at least one further surface; receive from a user via theat least one user input device an annotation input comprising a firstpart and a second part, wherein at least the second part extends beyondthe surface of interest; and draw the annotation in thecomputer-generated scene such that the first part is drawn on thesurface of interest and the second part are drawn on an imaginarysurface defined relative to the surface of interest.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: displaying in virtual reality acomputer-generated scene; obtaining a movement command from a real-worldphysical movement of a user, the movement command corresponding to amovement of a virtual body; and adjusting the movement of the virtualbody in dependence on an effect of gravity in the computer-generatedscene and/or in dependence on the presence of at least one object withinthe computer-generated scene that inhibits the movement of the virtualbody, wherein the adjusting of the movement is such that the adjustedmovement of the virtual body does not correspond with the real-worldphysical movement of the user.

Providing movement of a virtual body based on a user's real-worldphysical movement may provide an intuitive method of movement around thevirtual scene. By including effects of gravity and/or interaction withobjects in the scene, it may be possible to avoid making movements inthe computer-generated scene that are not possible in the real-lifeenvironment represented in the scene. For example, a virtual head maynot be able to float in mid-air, and a virtual body may not be able towalk through objects. Such constraints may provide increased realism inthe scene.

A combination of a virtual head and virtual body may provide a movementthat better simulates a physical movement than if only movement ofvirtual head were to be considered. For example, in some circumstances avirtual body or virtual body part may collide with an object in thescene, when the virtual head would not have collided with that object.

The virtual body may comprise at least one of a torso, an arm, a leg.The virtual body may represent at least part of a human form. Thevirtual body may be a simplified representation of at least part of ahuman form. The virtual body may comprise at least one geometricalshape. The virtual body may be one, two, or three-dimensional. Thevirtual body may be articulated.

If movement of only the virtual head (or a head and hands) were to beconsidered, it may be the case that a movement command would cause avirtual head and/or virtual body to become suspended in mid-air. Forexample, the user may request to step off a surface such as a table orbalcony, and the system may allow them to do so. The virtual head and/orvirtual body may be allowed to move to a viewing position which wouldnot be possible in a real-life structure represented in the scene. Byconsidering a virtual body, in some circumstances the user may not beallowed to perform some movements that would not be possible in thereal-life structure.

A movement of a virtual head may be adjusted in dependence on themovement of the virtual body. Adjusting the movement of the virtual headin dependence on the movement of the virtual body may comprise at leastone of reducing the movement of the virtual head, changing a trajectoryof the virtual head, changing an angle of the virtual head.

The movement of the virtual body may comprise a movement from a firstviewing position to a second viewing position.

The user may issue a movement command that requests the virtual head tomove from a first viewing position to a second viewing position. Inmoving from the first viewing position to the second viewing position,the virtual body may be inhibited by an object in the computer-generatedscene. Since the virtual body is coupled to the virtual head, theinhibition of the movement of the virtual body may reduce the movementof the virtual head. The resulting movement of the virtual head mayterminate at or near the object that inhibited the movement of thevirtual body.

By considering the movement of the virtual body, the user may beprevented from moving into a position that is impossible in real life bya representation of an object that would prevent them from doing so inreal life, for example by interaction with a representation of a wall orbalustrade in the computer-generated scene.

The virtual body may be deflected by the at least one object, causing atrajectory of the movement of the virtual head to change.

The first viewing position may be at a greater height in the scene thanthe second viewing position. The adjusting of the movement of thevirtual head may comprise adjusting the movement of the virtual head torepresent falling under gravity.

If the movement of the virtual body is affected by gravity, the movementof the virtual head may be adjusted to represent the action of gravityon the body. A position or orientation of the virtual head may bechanged.

The first viewing position may be at an elevated part of the scene. Forexample the first viewing position may be representative of the userstanding on a table or platform. The second viewing position may be at aless-elevated part of the scene. For example, the second viewingposition may be representative of the user standing on the floor.

The adjusting of the movement of the virtual head may comprise adjustingthe movement of the virtual head to represent a constraint on thevirtual body due to the presence of the at least one object, theconstraint preventing movement of the virtual head to the second viewingposition.

The at least one object may comprise at least one of a wall, abalustrade, a rope, a barrier.

The movement of the virtual body may be in dependence on a movement of avirtual head. The movement of the virtual body may be in dependence on amovement of at least one virtual hand.

The real-world physical movement of the user may comprise a movement ofthe user's head.

The virtual head may be prevented from moving into a position that isimpossible in real life. The virtual head may be prevented from movingby an object that interacts with the virtual body, but does not interactdirectly with the virtual head.

The user may walk in real life, and the user's walking may be translatedinto a movement command. The real-world movement may comprise a gesture.The real-world movement may comprise a movement of any appropriate partof the user's body, for example, a head, a torso, an arm, a leg, a handor a foot.

The obtaining of the real-world physical movement may be via a userinput device. The user input device may comprise at least one of a bodysuit, a tracking device, a camera, body recognition software.

The real-world movement may correspond to a movement that would not bepossible in a structure that is represented in the computer-generatedscene. For example, the real-world movement may correspond to steppinginto thin air or walking through a wall. By adjusting the movement, amovement may be obtained that makes sense in the computer-generatedscene. The movement may be adjusted to represent a movement that wouldbe possible in the structure. For example, if the user attempts to moveinto mid-air, the user may fall. If the user attempts to cross a barriersuch as the balustrade of a balcony, the user may be restrained. Theuser may be able to look over the balustrade, but may not be able towalk through the balustrade.

The movement command may comprise a teleport command from the firstviewing position to the second viewing position.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen, at least one user input device and at least oneprocessor configured to: display in virtual reality on the at least onedisplay screen a computer-generated scene; obtain a movement commandfrom a real-world physical movement of a user, the movement commandcorresponding to a movement of a virtual body; and adjust the movementof the virtual body in dependence on an effect of gravity in thecomputer-generated scene and/or in dependence on the presence of atleast one object within the computer-generated scene that inhibits themovement of the virtual body, wherein the adjusting of the movement issuch that the adjusted movement of the virtual body does not correspondwith the real-world physical movement of the user.

In another aspect of the invention, which may be provided independently,there is provided a method comprising: displaying in virtual reality acomputer-generated scene comprising an interactive virtual object, suchthat the interactive virtual object appears to be at a distance from auser; receiving from the user a remote selection of the interactivevirtual object; in response to the remote selection, displaying avirtual manipulation indicator overlaid with or adjacent to theinteractive virtual object, wherein the virtual manipulation indicatoris indicative of the interactive virtual object being in an active modein which the interactive virtual object can receive manipulationcommands; receiving a manipulation command from the user; and inresponse to the manipulation command from the user, operating or movingthe interactive virtual object, such that the interactive virtual objectappears to be operated or moved at a distance from the user.

Allowing operation of an interactive virtual object at a distance maymake it easier for a user to make changes in the appearance of thecomputer-generated scene. The user may not have to get near to theinteractive virtual object in the computer-generated scene in order tooperate it.

The interactive virtual object may be representative of a real controlwithin the computer generated scene, and operating the interactivevirtual object may allow the user to observe an effect of operating thatreal control.

The display of the virtual manipulation indicator may allow the user tounderstand which interactive virtual object is currently active.Therefore, the user may understand that any manipulation command will bedirected to that active object.

The interactive virtual object may comprise at least one of a lightingcontrol, a light switch, a door control, a door handle, a sound control,a window control, a window covering control, a heating control, an airconditioning control.

An interactive virtual object may represent any control within thecomputer-generated scene that in real life would be operable by aperson. An interactive virtual object may represent a control that isonly present in the computer-generated scene and would not be present ina real-life version of the structure represented in thecomputer-generated scene. An interactive virtual object may represent amoveable item, for example an item of furniture.

The virtual manipulation indicator may comprise a virtual hand.

The virtual hand may appear to be detached from the user. The virtualhand may appear not to be attached to a body. The virtual hand mayappear to be floating in space. The virtual hand may be at leastpartially transparent. The operating or moving of the interactivevirtual object may appear to be performed by a manipulation of thevirtual hand.

The operating or moving of the interactive virtual object may appear tobe performed by an action of the virtual manipulation indicator.

The display of an action of the virtual manipulation indicator may makeit clear how the interactive virtual object is controlled. For example,the virtual manipulation indicator may appear to switch a switch, rotatea knob, or slide a slider. The display of an action of the virtualmanipulation indicator may provide an intuitive display to the user. Thedisplay of an action of the virtual manipulation indicator may providean display that is representative of a real-life control method.

The remote selection may be provided by the user by indicating theinteractive virtual object using an input device.

The input device may comprise at least one of a handheld input device, acontroller, a games controller, a wand.

The manipulation command may be provided by the user using the or aninput device. The manipulation command may be provided using the inputdevice, for example by pressing a button on the input device. Themanipulation command may be provided by a physical movement of the user,for example by a gesture.

An action of the virtual manipulation indicator may mimic an action ofthe input device. An action of the virtual manipulation indicator maymimic an action of the user.

The mimicking of the input device may cause the operation of theinteractive virtual device to be intuitive to the user. A movement ofthe input device in space may cause a corresponding movement of thevirtual manipulation device in virtual space. The input device maycomprise, for example, a switch, button, knob or slider. The operationof the switch, button, knob or slider may be mimicked by the virtualmanipulation indicator to control an interactive virtual object that isrepresentative of a switch, button, knob or slider. For example, thevirtual manipulation indicator may be a virtual hand, and the virtualhand may move as if controlling a switch, button, knob or slider.

The remote selection may be provided by the user by pointing at theinteractive virtual object.

The pointing may comprise pointing the input device. The pointing maycomprise pointing using a hand of the user.

An apparent distance between the user and the interactive virtual objectmay be greater than 1 metre, optionally greater than 5 metres, furtheroptionally greater than 10 metres.

It may be more convenient for the user to be able to interact with theinteractive virtual object at a distance (for example a distance of 10metres or more) than to have to travel (for example, walk or teleport)to a position closer to the interactive virtual object.

The or a structure represented in the computer-generated scene maycomprise at least one of an architectural structure, a building, ahouse, an apartment building, a residential building, a commercialbuilding, an office building.

In a further aspect of the invention, which may be providedindependently, there is provided an apparatus comprising at least onedisplay screen and at least one processor, wherein the processor isconfigured to perform a method as described or claimed herein and todisplay in virtual reality the computer-generated scene on the at leastone display screen.

In another aspect of the invention, which may be provided independently,there is provided a computer program product comprising computerreadable instructions that are executable by a processor to perform amethod as claimed or described herein.

There may be provided a method or system substantially as describedherein with reference to the accompanying drawings.

Features in one aspect may be provided as features in any other aspectas appropriate. For example, features of a method may be provided asfeatures of an apparatus and vice versa. Any feature or features in oneaspect may be provided in combination with any suitable feature orfeatures in any other aspect.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the invention are now described, by way of non-limitingexample, and are illustrated in the following figures, in which:—

FIG. 1 is a schematic illustration of an apparatus in accordance with anembodiment;

FIG. 2 is a flow chart illustrating in overview a method of anembodiment comprising varying a time parameter;

FIG. 3 is an illustration of a screen shot representative of a firsttime of day;

FIG. 4 is an illustration of a screen shot representative of a secondtime of day;

FIG. 5 is a flow chart illustrating in overview a method of anembodiment comprising obtaining representations of a geographical areafrom photographic data;

FIG. 6 is an illustration of a screen shot showing a first and secondlocation on a structure;

FIG. 7 is an illustration of a screen shot showing a view from the firstlocation on the structure;

FIG. 8 is an illustration of a screen shot showing a view from thesecond location on the structure;

FIG. 9 is a flow chart illustrating in overview a method of anembodiment in which a user moves by selecting a location on a miniaturemodel;

FIG. 10 is an illustration of a screen shot showing a miniature model onwhich is marked a current location and a selected location;

FIG. 11 is an illustration of a screen shot of the selected location;

FIG. 12 is an illustration of a screen shot of the selected locationwith the miniature model displayed;

FIG. 13 is a flow chart illustrating in overview an annotation method;

FIG. 14 is an illustration of a screen shot showing a first part of anannotation;

FIG. 15 in an illustration of a screen shot showing a second part of anannotation;

FIG. 16 is a flow chart illustrating on overview a movement method;

FIG. 17 is an illustration of a screen shot showing a first viewingposition on a table and a second viewing position below the table;

FIG. 18 is an illustration of a screen shot viewed from the firstviewing position on the table;

FIG. 19 is an illustration of a screen shot viewed from the secondviewing position below the table;

FIG. 20 is an illustration of a screen shot showing a balcony;

FIG. 21 is an illustration of a screen shot from a viewing positionbeside the balcony;

FIG. 22 is an illustration of a screen shot from a viewing positionleaning over the balcony;

FIG. 23 is a flow chart illustrating in overview a method of interactingwith a remote object;

FIG. 24 is an illustration of a screen shot showing a remote lightswitch, a remote slider, and a controller;

FIG. 25 is an illustration of a screen shot showing a virtual hand nextto the remote light switch; and

FIG. 26 is an illustration of a screen shot showing the virtual handnext to the remote slider.

FIG. 1 is a schematic illustration of a virtual reality apparatus 10 inaccordance with an embodiment. The virtual reality apparatus 10comprises a virtual reality headset 20, computing apparatus 30, andinput device 40.

In the present embodiment, virtual reality headset 20 is a head-mounteddisplay (which in this embodiment is a HTC Vive virtual reality headset)configured to be worn on a user's head. Virtual reality headset 20comprises two screens 22 for displaying visual content to a user's eyessuch that the user perceives the visual content as being in threedimensions. Virtual reality headset 20 may also comprise opticalcomponents (not shown), for example optical components for making thescreens 22 appear to be further from the user than they actually areand/or optical components for correcting the image presented on thescreens 22. In other embodiments, any suitable display screen or screensand any suitable optical components may be used.

Virtual reality headset 20 further comprises a sensor or sensors 24configured to determine a position of the virtual reality headset 20.The position of the virtual reality headset 20 may be considered to berepresentative of a position of the user's head. In the presentembodiment, the sensors 24 comprise an accelerometer and a gyroscope. Inother embodiments, any suitable type and number of sensors 24 may beused. In further embodiments, no sensors 24 may be used. In someembodiments, a position of the virtual reality headset 20 is determinedusing one or more sensors or other components that are not part of thevirtual reality headset 20, for example by using a camera or othertracking device that monitors a position of the virtual reality headset20 from a distance.

Virtual reality headset 20 may comprise any suitable virtual reality oraugmented reality headset, for example an Oculus Rift or HTC Vivevirtual reality headset. Virtual reality headset 20 may comprise asmartphone headset, for example Google Cardboard.

In the present embodiment, virtual reality headset 20 is connected tocomputing apparatus 30 by a wireless connection. In other embodiments,virtual reality headset 20 may be connected to computing apparatus 30 byany suitable wired or wireless connection.

In the present embodiment, computing apparatus 30 is a personal computer(PC). In other embodiments, computing apparatus 30 may be any suitablecomputing apparatus or combination of computing apparatuses. Computingapparatus 30 comprises a processor 32 that is configured to provideimages for display on screens 22 of virtual reality headset 20. In thepresent embodiment, the processor 32 is configured to provide a livefeed of images, for example a feed of images at at least 45 frames asecond. In some circumstances, virtual reality applications may use ahigher frame rate than non-virtual reality media. Such a higher framerate may be a component of the virtual reality system feeling immersiveto the user.

The processor 32 comprises a memory 34 configured to store arepresentation of a three-dimensional scene and a graphics processingunit 36 configured to produce images of the three-dimensional scene fordisplay. In some embodiments, features of a virtual realityrepresentation provided by the processor 32 may correspond to one ormore features described in the Oculus Rift Best Practices document,https://developer3.oculus.com/documentation/intro-vr/latest/concepts/bp_intro/.

The processor 32 may also comprise scene creation circuitry configuredto create the representation of the three-dimensional scene. Thethree-dimensional scene may be representative of a real-worldenvironment, for example the interior of a building. The scene maycomprise multiple computer-generated objects, for example objectsrepresentative of walls, floors, ceilings, furniture and accessories.Each object may be modelled independently. The objects may berepresented in 3D. For example, objects may be represented as polygonalmeshes, which may also be referred to as a wire-frame representation.Objects may be represented as a combination of geometrical shapes.Objects may be represented as a combination of surfaces. Objects may bedefined in a three-dimensional coordinate system of the scene.

The graphics processing unit 36 may comprise lighting circuitryconfigured to produce lighting effects and/or rendering circuitryconfigured for image rendering. Images of the three-dimensional scenemay be rendered as if viewed from a viewing position having a coordinateposition that is inside the three-dimensional scene. When an image ofthe scene is rendered, it may show only a part of the scene, for examplean individual room of a building. Objects within the scene may berendered with any suitable image effects. For example, objects may berendered as opaque or as at least partially transparent. Objects may berendered with particular colours and/or textures, which may or may notbe realistic. Lighting effects may simulate lighting from light fixturesand/or external light, for example light arriving through windows. Anysuitable lighting methods may be used, for example ray casting or globalillumination. The lighting circuitry may simulate effects of reflectionand/or refraction of light.

The graphics processing unit 36 is configured to produce a sequence ofimages in real time which provide the effect of the user beingpositioned in the three-dimensional scene. The images are rendered so asto provide a three-dimensional display by providing different input toeach screen 22 of the virtual reality headset 20, giving the user theimpression that they are viewing a three-dimensional environment inperspective view. The user may have an experience of being immersed inthe virtual environment. A realistic effect of being in the environmentmay be provided to the user.

The user may move through the computer-generated scene by various means,for example by using the user input device 40 as described below. Theuser may move in the coordinate space of the three-dimensional scenefrom a first viewing position to further viewing positions. As the usermoves through the scene, the rendered images change in response to thechange in viewing position within the scene. For example, the user mayview the objects in the scene from different angles and/or distances.

In some embodiments, objects within the three-dimensional scene may bemoveable. Some objects may be manipulated by a user. Properties of anobject may be changed, for example a size, colour, transparency ortexture of an object. The processor 32 may comprise object evolutioncircuitry configured to make changes to objects and/or record thecurrent status of objects. The processor 32 may comprise physicscircuitry configured to simulate physical effects on objects, forexample by making objects obey gravity. The physics circuitry maycontrol interactions of objects with other objects, for example bycausing an object to be stopped or deflected when it contacts anotherobject, rather than passing through that object. In a games context, theprovision of realistic physical interactions may be referred to as gamesphysics.

The scene may be regenerated over time in response to inputs from theuser. For example, the user may change the position of an item withinthe scene or annotate the scene. Object properties may be recalculatedto provide an updated scene, and images of the updated scene may berendered by the graphics processing unit 36.

The processor 32 may comprise input circuitry configured to processinputs from the input device 40 and/or from other input devices. Theprocessor 32 may comprise movement circuitry configured to monitormovement of a user, for example in dependence on input from sensors 24of the virtual reality headset 20 and/or orientation circuitryconfigured to monitor an orientation of the user, for example independence on input from sensors 24.

Computing apparatus 30 (for example, input circuitry of the processor32) is configured to receive inputs from the sensors 24 and from theuser input device 40. Computing apparatus 30 (for example, the graphicsprocessing unit 36) is configured to provide images generated independence on the inputs from the sensors 24 and from the user inputdevice 40. In further embodiments, computing apparatus 30 is configuredto receive further inputs, and to provide images generated in dependenceon those further inputs. For example, the computing apparatus 30 mayfurther receive further inputs from at least one further input device(for example, a mouse, keyboard or trackball); from further sensors orother components; from a camera or other tracking device; and/or from atleast one further computing apparatus.

Computing apparatus 30 also includes a hard drive and other componentsincluding RAM, ROM, a data bus, an operating system including variousdevice drivers, and hardware devices including a graphics card. Suchcomponents are not shown in FIG. 1 for clarity.

Input device 40 is configured to be used by the user for inputtingcommands to computing apparatus 30. In the present embodiment, inputdevice 40 is a wand comprising a plurality of buttons 42 that may bepressed by the user to provide input commands. In other embodiments, theinput device 40 may be any handheld device, for example a wand or gamescontroller. In further embodiments, the input device 40 may be anysuitable input device, for example a keyboard, mouse or joystick.

In the present embodiment, input device 40 is connected to computingapparatus 30 by a wireless connection. In other embodiments, inputdevice 40 may be connected to computing apparatus 30 and/or to virtualreality headset 20 by any suitable wired or wireless connection.

In the present embodiment, the computing apparatus 30 provides visualcontent to the screens 22 of the virtual reality headset 20. The userwears the virtual reality headset 20 and views the visual content viathe screens 22. In further embodiments, the computing apparatus 30provides visual content to one or more display screens that are not partof the virtual reality headset, for example 3D television screens orcomputer monitors. In some embodiments, no virtual reality headset 20 isused. In some embodiments, one user views the visual content via thevirtual reality headset 20 and another user views the visual content viaone or more further screens that are not part of the virtual realityheadset 20. In some embodiments, multiple users may each view the visualcontent through a respective virtual reality headset 20.

The apparatus of FIG. 1 is configured to perform the processes of FIGS.2, 5, 9, 13, 16 and 23 , each of which is described in turn below.

In each of the processes of FIGS. 2, 5, 9, 13, 16 and 23 , the visualcontent provided to the user is representative of an architecturalstructure in a geographical location. For example, the visual contentmay be representative of a proposed building that has not yet beenbuilt, or of proposed modifications to an existing building. The usermay be an architectural client, architect, surveyor, planner, builder,or any other appropriate user. The use of virtual reality may allow theuser to view the building as if it were already built or modified.Virtual reality may allow an immersive experience in which the user canmove around a proposed building. In some embodiments, a virtual realityrepresentation of the building may allow the user to see the building incontext, for example placed in its proposed surroundings.

In other embodiments, the virtual content may be representative of anysuitable structure. The methods described below may be used in, forexample, design, gaming, or educational or training fields.

FIG. 2 is a flow chart illustrating in overview a method of anembodiment. At stage 50 of the flow chart of FIG. 2 , the processor 32generates a computer-generated scene representative of an architecturalstructure in a geographical location, and of a geographical areasurrounding the geographical location. In the present embodiment, thestructure is a house that has not yet been built, and the geographicallocation is a proposed site for the house. The geographical areasurrounding the geographical location is based on the currentsurroundings of the proposed site for the house. In further embodiments,the geographical area surrounding the house may not be shown.

The processor 32 represents the house as a plurality ofcomputer-generated objects, for example, objects representative ofwalls, floors, ceilings, and furniture. The processor 32 may alsorepresent at least part of the geographical area surrounding theproposed site for the house as a plurality of computer-generatedobjects, for example objects representative of other buildings.

At stage 52, the processor 32 displays the computer-generated scene invirtual reality by rendering a series of images of the scene that arerepresentative of the scene as viewed from a viewing position. Theviewing position may change as a user moves through the scene in virtualreality (for example, by inputting movement commands or by makingphysical movements).

In the display of stage 52, the processor 32 displays the scene as ifviewed at a first time of day. The first time of day may be stored inthe processor 32, automatically generated, or selected by a user. In thepresent embodiment, the processor 32 determines a first sun positionbased on the first time of day and geographical location, and uses thedetermined sun position to determine parameters of sunlight which areused in the display of the scene of stage 52, for example a directionand strength of sunlight. In some embodiments, the processor 32determines the first sun position based on a time of year in addition tothe time of day and geographical location. In some embodiments, thegeographical location used may be an approximate or generic geographicallocation instead of a geographical location at which the building is tobe placed.

FIG. 3 is an illustration showing the scene at the first time of day,which in FIG. 3 is 9:17 am. FIG. 3 is a screen shot shown as if viewedby a user who is wearing a virtual reality headset 20. The display ofthe scene comprises a time display 60 indicating the first time of day,9:17 am. Since the first time of day is during daytime, part of thegeographical area surrounding the house is visible through a window ofthe house in FIG. 3 .

The user is holding a handheld input device 40, which is represented inthe display shown in FIG. 3 by virtual handheld input device 62. Thehandheld image device 40 comprises a button 44 (represented by button 64in FIG. 3 ) which is designated for time control. In other embodiments,any suitable input device may be used.

At stage 54 of the process of FIG. 2 , the processor 32 receives aninput from the user via button 44 of the handheld device 40. The inputis representative of a variation in the time of day. In the presentembodiment, the user indicates a variation in time of day by pressingdown button 44. Pressing down button 44 causes the time shown on thetime of day display 60 to increase. The time shown on the time of daydisplay 60 stops decreasing when the user stops pressing the button. Inother embodiments, the user may operate any suitable user input tool toprovide an input that is representative of an increment in time, or of aparticular selected time. For example, the user may hold down a button,slide a slider, or press a pressure-sensitive device. The user may typein a time or select a time from a list.

At stage 56, the processor 32 displays the scene as if viewed at asecond time of day. The second time of day is determined based on theuser input. In the present embodiment, the second time of day is thetime of day at which the user stopped pressing the button, which in theexample shown is 10:16 pm. The processor 32 determines a second sunposition based on the second time of day and geographical location. Inthis example, the processor 32 determines that the sun is below thehorizon and so no sunlight is present in the scene.

FIG. 4 is an illustration showing the scene at the second time of day,10:16 pm. FIG. 4 is a screen shot shown as if viewed by a user who iswearing a virtual reality headset 20. The display of the scene comprisesthe time display 60 which now indicates the second time of day, 10:16pm. The display is dark and the geographical area cannot be seen throughthe window.

In the present embodiment, the scene is shown as dark when it is nighttime (for example, at 10:16 pm as shown in FIG. 4 ). In otherembodiments, the displayed scene shows lights in the geographical area(for example, simulated house lights or street lights) and/or within thehouse. For example, if the house is shown at 10:16 pm, it mayautomatically be shown with some interior lighting turned on.

In some embodiments, the processor 32 calculates a position of the moonand/or positions of stars or planets based on a time of day and/or timeof year and geographical location, and displays the moon, stars orplanets at times that the moon, stars or planets would be visible fromthe geographical location.

Updating the computer scene in dependence on a variation of time of daymay enable the user to vary on command an appearance of at least part ofthe house in the computer-generated scene. In the present embodiment,the processor 32 receives the input indicating the change in time andthen changes the display to represent the change in time, withoutshowing the scene at intermediate times. In other embodiments, thedisplay of the scene changes in real time while the user presses thebutton. The display shows the scene at a succession of times between thefirst time of day and the second time of day. For example, the user maysee the sun rise and/or set while the user presses the button to moveforward through time. The display may change in a manner that is similarto a time-lapse video.

By holding down a button 44 on the controller 40, the user can directlycontrol the time of day in the virtual world. The time of day in thevirtual world is indicated by the time display (which may be referred toas a heads-up display time display or HUD time display). The time of dayin the virtual world may also be indicated by in-world lighting.

In the embodiment described above with reference to FIGS. 2 to 4 , theuser provides an input representative of a change in time of day. Inother embodiments, the user provides an input representative of a changein time of year. In further embodiment, the user provides an inputrepresentative of a change in at least one weather parameter.

A calendar display indicating a time of year may be displayed to theuser. A map showing the geographical location may be displayed to theuser. An icon or other display showing weather conditions may bedisplayed to the user.

In some embodiments, the processor 32 receives a user inputrepresentative of a variation in at least one weather parameter. Forexample, the user input may be provided by pressing a button on handheldinput device 40 (which may or may not be the same button 44 as a buttonused to provide time input), selecting a weather parameter from a list,typing in a text input, or by any suitable method. The processor 32processes the user input and makes a change in the weather representedin the computer-generated scene in dependence on the user input. Forexample, the processor 32 may change an extent of cloud coverage, a typeof precipitation, a rate of precipitation, an amount of fog or an amountof smog.

In further embodiments, the processor 32 determines at least one weatherparameter based on a time of day and/or on a time of year. For example,the processor 32 may obtain typical weather conditions for a selectedtime of day and/or time of year and display the computer-generated scenehaving typical weather.

Allowing the user to change a type of weather may allow the user to seehow the house looks in different weather conditions. Different weatherconditions may affect an amount or quality of light coming into thehouse. An updating of the weather conditions may be provided in realtime.

In other embodiments, the method of FIG. 2 may be used for a virtualreality application that comprises displaying any architecturalstructure. In further embodiments, the method of FIG. 2 may be used forany suitable virtual reality application, for example for gaming. Themethod of FIG. 2 may be used for virtual maps and/or virtual tourism.Any appropriate location may be represented in virtual reality withcurrent or predicted lighting or weather conditions.

FIG. 5 is a flow chart illustrating in overview a method of anembodiment. At stage 70 of FIG. 5 , the processor 32 obtainsphotographic data representative of a geographical area around astructure of interest. In the present embodiment, the structure ofinterest is a house that has not yet been built. The geographical areais an area around a proposed site of the house.

In the present embodiment, the photographic data is representative of ageographical area that surrounds the house on all sides and extends tothe furthest distance that can be seen from the house. In otherembodiments, the geographical area may be any geographical area thatsurrounds or is adjacent to the structure of interest.

In some embodiments, the geographical area is an area within a fixedthreshold distance from the structure of interest. In some embodiments,the geographical area includes all structures that are viewable from thehouse.

In some embodiments, the extent of the geographical area is different indifferent directions. For example, the house may not have windows on oneside, or may look out onto a nearby building on that side. In such acase, the extent of the geographical area on that side of the house maybe limited. In some circumstances, a view from the house may be blockedin one direction by a geographical feature, for example a hill, and thegeographical area may extend only as far as that geographical feature.

In the present embodiment, the geographical area is determined manually.In other embodiments, the geographical area may be determinedautomatically or semi-automatically.

In the present embodiment, the photographic data is aerial data that hasbeen obtained using a drone. A large range of imagery is captured via adrone over multiple flights in one short time period (for example, atime period of an hour). The photographic data comprises a plurality oftwo-dimensional photographic images. The photographic data includesimages of all of the geographical area. At least some of thetwo-dimensional images may have been taken from an aerial location nearthe house. In the present embodiment, the aerial data is takenspecifically to be used for the virtual display of the house. In otherembodiments, existing aerial data may be used.

The photographic data comprises imagery from multiple angles andpositions. In the present embodiment, the photographic data isdrone-captured aerial photography which captures imagery from positionsincluding positions that are not currently accessible via a structure.The photographic data comprises high-resolution colour photographs.

At stage 72, the processor 32 divides the geographical area into a firstpart near the structure of interest, and a second part that is furtheraway from the structure of interest than the first part. In the presentembodiment, the geographical area surrounds the structure of interest.The first part of the geographical area is the part of the geographicalarea that is within a threshold distance of the structure of interest,for example within 400 metres of the structure of interest. The secondpart of the geographical area is the part of the geographical area thatis outside the threshold distance.

In other embodiments, the threshold distance varies with angle aroundthe house. In further embodiments, a criterion other than thresholddistance is used to divide the geographical area into first and secondparts. For example, the geographical area may be divided based on thepresence of structures in different parts of the geographical area, oron characteristics of those structures.

In the present embodiment, the geographical area comprises a pluralityof structures, including other houses and other buildings. Eachstructure of the plurality of structures is assigned to either the firstpart of the geographical area or the second part of the geographicalarea. In the present embodiment, the structures are assigned based ondistance from the house. Structures within 400 metres of the house areassigned to the first part and structures more than 400 metres from thehouse are assigned to the second part. In other embodiments, structuresmay be assigned to the first or the second part based on their size orimportance. For example, the first part may comprise structures that arewithin a threshold distance from the house and some large and/orimportant structures that are outside the threshold distance.

In some embodiments, the geographical area is divided into a first partand second part before the photographic data is obtained. For example,the geographical area may be divided based on known mapping data.

At stage 74, the processor 32 processes the photographic data to obtaina three-dimensional representation of the first part of the geographicalarea. The processor 32 thereby obtains a three-dimensionalrepresentation of structures that are near to the house, for examplenearby buildings.

Any suitable processing method may be used to obtain thethree-dimensional representation, for example any suitablephotogrammetry method. In some embodiments, the processing uses furtherdata in addition to the photographic data. The further data maycomprise, for example, GPS data, mapping data, laser data or radar data.

At stage 76, the processor 32 processes the photographic data to obtaina two-dimensional representation of the second part of the geographicalarea. In the present embodiment, the two-dimensional representation is apanoramic image of the second part of the geographical area as if viewedfrom a viewing point that is within or near the house.

In other embodiments, the processing of the photographic data may beperformed using any suitable computing apparatus, which may or may notbe computing apparatus 30. In one embodiment, a PC is used to run thevirtual reality, while a further computing apparatus processes thephotographic data by performing photogrammetry. The further computingapparatus performs the photogrammetry in an offline process usingphotogrammetry tools. The generated geometry and textures from thephotogrammetry process are used in stage 78.

At stage 78, the processor 32 generates a computer-generated scenerepresentative of the house, and of the geographical area surroundingthe geographical area. In the computer-generated scene, the first partof the geographical area is represented by the three-dimensionalrepresentation obtained at stage 74, and the second part of thegeographical area is represented by the two-dimensional representationobtained at stage 76. The processor 32 displays the scene in virtualreality by rendering a series of images of the scene that arerepresentative of the scene as viewed from a viewing position.

The viewing position from which the scene is rendered may or may not bethe same as the viewing position from which the two-dimensionalrepresentation was determined. However, the second part of thegeographical area may be far enough away from the house that a smallchange in viewing position (for example, a change from a first viewingposition within or near the house to a second viewing position within ornear the house) may not change significantly an appearance of structuresin the far distance. Therefore, the two-dimensional representation mayprovide an adequate representation of structures in the far distancewhen viewed from any viewing position that is within or near the house.

FIG. 6 is an illustration showing a computer-generated scene in whichtwo viewing positions 80, 82 are marked. The scene is representative ofa balcony, and the two viewing positions 80, 82 are different points onthe balcony. In practice, the scene is rendered as if viewed from adistance above the marked viewing positions 80, 82, the distancecorresponding to a user's height.

FIG. 7 is an illustration showing a view of the computer-generated sceneas viewed from the first viewing position 80. Two buildings 90, 92 arehighlighted in FIG. 7 by outlining. Building 90 is within the thresholddistance from the house, and so is in the first part of the geographicalarea. Building 90 is represented in 3D geometry. Building 92 is outsidethe threshold distance from the house, and so is in the second part ofthe geographical area. Building 92 is represented in a static panoramicimage.

FIG. 8 is an illustration showing a view of the computer-generated sceneas viewed from the second viewing position 82. The same two buildings90, 92 as were highlighted in FIG. 7 are also highlighted in FIG. 8 . Itmay be seen that, by changing viewing position from the first viewingposition 80 to the second viewing position 82, there is a change in anapparent position of building 92 relative to building 90. The change inapparatus position may be referred to as a parallax effect.

In FIGS. 7 and 8 , the large highlighted building 90 is 3D geometry andthe smaller highlighted building 92 is a static photograph. The relativepositions of buildings 90 and 92 change due to parallax between FIGS. 7and 8 . However, the distant building 2 does not move relative to evenfurther objects. All buildings in FIGS. 7 and 8 may be considered tohave a very similar visual appearance.

In the embodiment described above with reference to FIGS. 5 to 8 , arepresentative view outside the property (in this embodiment, the house)is created. The representative view outside the house is created using3D geometry of nearby buildings, which in this embodiment are buildingsin the first part of the geographical area. Buildings in the first partof the geographical area are within a threshold distance from the house.

In some circumstances, using a 3D model into the far distance may beimpractical for performance reasons. Using a separate solution for nearbuildings than for far buildings may in some circumstances risk a visualmismatch. However, in this embodiment, we take a single set ofdrone-captured imagery from around the property, and create a 3D modelof near buildings, and a single static panoramic image of the fardistance. Because the 3D model and panoramic image are sourced from thesame data, they may provide a close visual match.

In some embodiments, one part of the photographic data is used togenerate 3D models and a different part of the photographic data is usedfor the 2D panorama. If the different parts of the photographic data aretaken using the same drone, in the same broad location, and at roughlythe same time of day, the 3D and 2D images may still match.

The technique of FIG. 5 may be used with very closely matched data, forexample data captured from a very close location at a similar time ofday with similar camera settings. In some embodiments, differentphotographic data is used to generate the 3D representation from thedata used to generate the 2D representation, but the photographic dataused is matched in time of day and/or settings.

In other embodiments, the method of FIG. 5 may be used for any suitablevirtual reality application, for example in games in which a real-worldenvironment is simulated. The method of FIG. 5 may be used for virtualmaps or virtual tourism.

FIG. 9 is a flow chart illustrating in overview a method of anembodiment. In the embodiment of FIG. 9 , a user moves around acomputer-generated scene by pointing a wand head at a location in aminiature model of a building. To move around the building, theminiature model of the building is displayed, and a controller is usedto point to the location that the user wishes to move to.

At stage 100, the processor 32 generates a computer-generated scenerepresentative of at least part of a house. The processor 32 displaysthe scene in virtual reality by rendering a series of images of thescene that are representative of the scene as viewed from a firstviewing position.

The computer-generated scene comprises a miniature model of the house.The scene is presented such that the miniature model of the houseappears to be displayed in a part of the house or in a space associatedwith the house. For example, the miniature model may be displayed in aroom of the house, or in an outdoor space such as a balcony or garden.

The miniature model of the house is presented at a first, smaller scale.A part of the house in which the miniature model appears to be displayedis displayed at a second, larger scale.

In other embodiments, the miniature model may be presented alone. Theminiature model may not be presented in a room of the house or in aspace associated with the house. For example, the miniature model may bepresented on a solid-colour background or as part of a navigationscreen.

FIG. 10 shows a miniature model 110 in a room of a house. Thecomputer-generated scene representative of the room of the house isdisplayed as if viewed from a viewing position within the room. In FIG.10 , the viewing position within the room is indicated in the miniaturemodel by a first circle 112. In an embodiment, the user's currentposition is indicated in the miniature model 110 by a green sphere. Theuser can therefore see in the miniature model a position in the housefrom which the user is viewing the house. The user may thereforeunderstand how the parts of the house that they can see from the viewingposition relate to other parts of the house. The user may simultaneouslysee representations of the house at two different scales

At stage 102, the user selects a location on the miniature model that isrepresentative of a further viewing position. The further viewingposition may be within the same room as the viewing position of stage100, within a different room of the house, or in a space associated withthe house.

In the present embodiment, the user selects the further viewing positionusing a virtual wand 114 (which corresponds to real-life input device40). The user selects the further viewing position by pointing the wand114 at the miniature model to select a location on the miniature model100. In other embodiments, any input device, for example any handheldinput device, may be used to select the further viewing position. Anysuitable method may be used to select the further viewing position. Insome embodiments, the further viewing position is selected using a gazeinterface in which the user provides a user input by looking at aspecific location and holding down a button for a period of time. Forexample, the user may keep the centre of their vision focused on aspecific room or location in the miniature model while holding a buttonto teleport to that location.

FIG. 10 shows the wand 114 of the present embodiment. Wand 114 is shownpointing to a selected location on the miniature model 110. The selectedlocation is indicated as a second circle 116. The second location isnext to the pool. In some embodiments, the second location is indicatedby a red sphere at the wand head, or by a red sphere at a position thatis pointed to by the wand head.

At stage 104, the user provides an instruction to teleport to the secondviewing position. In the present embodiment, the user provides theinstruction to teleport by pressing a button on the wand 114. In otherembodiments, the user may provide the instruction to teleport using anysuitable input method.

At stage 106, the processor 32 displays the computer-generated scene (oranother computer-generated scene) as if viewed from the further viewingposition, thereby teleporting the user to the further viewing position.The further viewing position is a position within the scene at thesecond, larger scale that corresponds to the selected location in theminiature model.

FIG. 11 is a computer-generated scene viewed from the location that wasselected in FIG. 10 using the wand 114 and represented by second circle116, i.e. from a location next to the pool.

In the present embodiment, the user selects a location in the miniaturemodel and the processor 32 displays the computer-generated scene as ifviewed from a further viewing position that corresponds to the selectedlocation. In other embodiments, the user selects a location in theminiature model and the processor displays the computer-generated sceneas if viewed from a viewing position in the structure that is selectedin dependence on the selected location in the miniature model. Forexample, the viewing position may be a viewing position from which it ispossible to look at a location in the structure that corresponds to theselected location in the miniature model.

At stage 108, the processor 32 displays the miniature model in thecomputer-generated scene of stage 106. In some embodiments, the displayof the miniature model is similar to the display at stage 100. Forexample, the miniature model may be viewed from the same viewing angle.In other embodiments, the display of the miniature model may bedifferent. For example, the miniature model may be viewed at a differentangle or at a different scale. In some embodiments, the display of theminiature model (for example, a display angle or scale) is controllableby the user.

FIG. 12 shows the scene as viewed from the further viewing position ofFIG. 11 with the miniature model 110 included in the scene. The furtherviewing position is indicated by circle 114. In some embodiments, theindication of the further viewing position changes colour to show thatthe scene is now being viewed from the further viewing position. Forexample, the further viewing position beside the pool may appear as ared sphere when it is selected (for example, in a display such as thedisplay of FIG. 10 ) and may appear as a green sphere when the scene isbeing viewed from the further viewing position (for example, in adisplay such as the display of FIG. 12 ).

In some embodiments, stage 108 is omitted. When the user teleports, theuser sees the scene as if viewed from the further viewing position,without the presence of the miniature model. In some such embodiments,the miniature model may be displayed in the scene at the user's request.

In some embodiments, stage 106 and stage 108 are performedsimultaneously or near-simultaneously. The user may see the miniaturemodel as soon as the user teleports to the further viewing position. Insome embodiments, the user controls when the miniature model is visible.For example, the user may toggle visibility of the miniature model onand off. The user may rotate the miniature model. As the user movesround a space in the computer-generated scene (for example, a room,balcony or garden), the user may also appear to move round the miniaturemodel.

In other embodiments, as the user moves round a space in thecomputer-generated scene, the miniature model may appear to moverelative to the scene, for example, such that the user always viewed theminiature model at a consistent angle.

In some embodiments, the miniature model is displayed permanently orsemi-permanently. The display of the miniature model may persistwhenever a movement or other change of display is made. The miniaturemodel may be displayed such that it minimises any interference with themain view of the scene. For example, the miniature model may besemi-transparent and/or positioned in the corner of the scene. It mayalways be possible to directly interact with the miniature model inorder to move around the structure.

After stage 108, the user may select another viewing position in theminiature model and teleport to that other viewing position using themethod described above with reference to stages 102 to 108.

The miniature model may provide a convenient and intuitive display inwhich a user may understand the context of a scene or part of a scenethat they are viewing. Navigating by pointing at the miniature model mayprovide a convenient movement mechanism around the house.

In further embodiments, the miniature model may represent more than theindividual building and/or grounds in which the user appears to bestanding. For example, the miniature model may represent furtherbuildings or nearby streets. In some embodiments, the movement mechanismof FIG. 9 may allow the user to teleport to any position in theminiature model.

In further embodiments, the method of FIG. 9 may be used to navigatearound any suitable structure. The method of FIG. 9 may be used in anysuitable virtual reality application. The method of FIG. 9 may be usedfor virtual maps or virtual tourism.

FIG. 13 is a flow chart illustrating in overview a method of anembodiment. The method of FIG. 13 may provide an annotation featurewhere annotation is projected onto a plane if a cursor moved off theobject being annotated.

At stage 120 of FIG. 13 , the processor 32 generates acomputer-generated scene representative of a structure. The structurecomprises a plurality of surfaces, for example a floor, walls, andsurfaces of objects such as furniture.

At stage 122, the user provides a first part of an annotation inputrepresentative of a first part of an annotation in the scene. In thepresent embodiment, the user uses a handheld input device as a virtuallaser pointer. When the user moves the handheld input device, a cursorrepresenting a light of the laser pointer moves across the scene. Thecursor may be considered to represent light which is projected onto thefirst surface that a light beam of the laser pointer intersects, i.e.the surface that is closest to the laser pointer along its line ofsight.

The user provides the first part of the annotation input by moving thecursor along a surface in the computer-generated scene, which in thisembodiment is the side of a table in the computer-generated scene. Theprocessor 32 determines that the surface on which the user starts todraw is the surface of interest. The initial use of the laser pointer onthe side of the table selects that side of the table as the surface ofinterest.

As the user moves the cursor along the side of the table, the processor32 draws a line onto the table in accordance with the movement of thecursor.

FIG. 14 shows a table 126 on which a line 128 has been drawn inaccordance with the movement of the cursor by the user.

At stage 124, the user provides a second part of an annotation inputrepresentative of a second part of an annotation in the scene. Thesecond part of the annotation input continues on from the first part. Insome circumstances, the user may not consider the annotation to have afirst part and a second part, but instead may consider the annotationinput to have been provided as a continuous movement.

In the present embodiment the user provides the second part of theannotation input by continuing to move the cursor beyond the edge of thetable, following a broadly rectangular path.

When the cursor reaches the edge of the table, the laser light that itrepresents may be considered to jump from the table to the floor besidethe table. At a location at which the laser light would move betweensurfaces, the processor 32 determines that there has been a transitionbetween surfaces.

In the present embodiment, the processor 32 determines that there hasbeen a significant distance change in the distance travelled by thelaser light. The processor 32 compares the distance change to athreshold value. In the present embodiment, the threshold value is 25cm. In other embodiments, the threshold value may be any suitablethreshold value, for example 10 cm or 50 cm. If the distance change isgreater than the threshold value, the processor 32 determines that therehas been a transition between surfaces. In other embodiments, anythreshold value may be used. Any method of determining a transitionbetween surfaces may be used.

If the processor 32 were to draw the annotation as if drawn by a laserpointer, the annotation would move to the next surface that the beam ofthe laser pointer contacts, which in the embodiment shown in FIG. 14 isthe floor 129 of the room shown in the computer-generated scene.

However, in this embodiment, the processor 32 draws the second part ofthe annotation as if it were drawn on an invisible plane aligned withthe surface of the table on which the first part of the annotation wasdrawn. The annotation may represent an extension to the table.

The invisible plane may be referred to as an imaginary surface. All thesurfaces in the scene are virtual surfaces. However, the invisible planeis not itself represented in the scene. In other embodiments, theprocesser 32 may define any imaginary surface relative to the surface ofinterest, which may or may not be a plane.

In the present embodiment, the invisible plane is determined using anormal vector taken from the normal of the surface of interest at afinal point of the first part of the annotation input before the laserpointer jumped to the floor. In other embodiments, any normal vector forthe plane may be used. A normal may be taken from the surface ofinterest at an initial point of annotation. A normal may be calculatedas a vector perpendicular to a line connecting two previous annotationpoints (for example, connecting the initial and final points ofannotation) and either horizontal or vertical.

In other embodiments, any method may be used for determining the planeor other imaginary surface.

FIG. 15 shows the first part 127 and second part 128 of the annotationwhich are both drawn on the plane of the side of the table 126. No partof the annotation is drawn on the floor 129.

If the annotation were only to be viewed from the first viewing positionat which the computer-generated scene were displayed when the user wasproviding the annotation input, it may not matter whether the annotationwere considered to be on a plane aligned with the table, or on thefloor. It may still be apparent to a viewer that the user intended toindicate an extension of the table. However, the position of theannotation may be relevant if the annotation is viewed from a differentviewing position, for example from a different angle.

In the present embodiment, the processor 32 displays thecomputer-generated scene and the annotation as if viewed from a secondviewing position, such that the computer-generated scene appears to havebeen rotated. The line has a fixed 3D position and hence will alwayschange appearance if the viewing position changes.

Since the second part of the annotation is in a plane aligned with theside of the table, the second part of the annotation still appears toextend beyond the table when viewed from any angle. For example, a usermay virtually walk round the table and view the annotation as protrudingfrom the end of the table no matter which viewing position is assumed bythe user. The annotation may have a consistent position relative to thetable in three dimensions. The maintaining of the position of theannotation in three dimensions may cause the appearance of theannotation from one viewing position (for example, side-on) may bedifferent from an appearance of the annotation from another viewingposition (for example, end-on).

In further embodiments, the processor 32 displays the annotation so thatit appears to be related to the surface of interest (for example, theside of the table), but so that it appears to change shape and/orposition in three dimensions as the user moves. For example, if theannotation comprises a square around a small object of interest, thesquare may always look square to the user no matter which angle it isviewed from.

In some embodiments, the annotation always maintains the same visualshape regardless of viewing angle. In some embodiments, a scale of thevisual shape is changed based on viewing distance. A centre of the shapemay be determined by taking an average of all points or by using aninitial drawing location. The annotation may then always be displayed asif it is facing the user, but pinned to the centre point. In oneembodiment, a circle is drawn around an object. The circle still appearsas a circle from all viewing locations.

A virtual laser pointer, or any other virtual annotation tool, may beused to draw onto geometry for annotation purposes. In some systems, bydefault, the drawing will take place on the first geometry to be hit onthe laser pointer's line of sight. This may made it hard to annotatesmall objects, or to draw around an object. In such cases, theannotation may look correct from the perspective of the personannotating, but not from other locations in the 3D world.

The plane-projection technique of FIG. 13 may detect when the laserpointer has jumped onto more distant geometry, and instead continue toannotate on a plane aligned with the previous geometry.

FIG. 14 shows a line being drawn onto the side of a table. By default,as that line moves right off of the table, it would appear on the floorbehind. Instead, by using the method of FIG. 13 , the line is drawn ontothe same plane as that of the side of the table, but extended beyond thetable itself.

In the example described above, the beam of the laser pointer jumps fromthe side of the table 126 to the floor 129. The system detects asignificant distance change in a short space of time, and hencedetermines that the surface has changed, and it that it should insteaddraw on an imaginary plane (instead of jumping to the floor 129).

Although in the example above the side of the table 126 is planar, insome embodiments the surface of interest may not be flat. The user maydraw around corners or on a curved object. The user may draw on anysurface that is continuous or near-continuous, whether or not it isflat. In such embodiments, the processor 32 determines a transition ifthe laser beam jumps by a significant distance, for example a distanceabove a threshold value. If the laser beam has jumped by a distancebelow the threshold value, the processor 32 continues to draw on thesurface on which the laser beam lands, and does not draw on an imaginaryplane or other imaginary surface. For example, in FIG. 14 , if the userwere to draw upwards instead of to the right, the line may bend onto thetop of the table.

In some embodiments, the imaginary surface may be a best fit to asurface of interest. The imaginary surface may or may not be alignedwith the surface of interest. For example, when annotating a spherethere may be no one plane aligned to that sphere. The processor 32 maychoose a plane aligned to the normal of the sphere at an initial, final,or average point of annotation.

In some embodiments, once the system has started drawing on an imaginaryplane, it then detects if the actual surface being hit by the lasercoincides, or nearly coincides, with the imaginary plane.

If the actual surface being hit by the laser coincides, or nearlycoincides, with the plane, the processor 32 draws the annotation suchthat it starts respecting the surface again and stops being limited tothe imaginary plane. This may allow the user to drawn on a curvedsurface, then jump to a plane, and then return to the original curvedsurface.

In the present embodiment, the method of FIG. 13 is appliedautomatically when annotating. The decision to draw on an imaginarysurface is automated based on a distance jump threshold. The processor32 assumes that if a user draws a line that starts on one surface andcontinues onto another surface at a distance greater than a thresholddistance, the user intended for the line to continue in the plane of thefirst surface. However, in other embodiments, the user may turn on andoff the method of FIG. 13 . A toggleable menu setting may be used. Theuser may provide a direct user input such as holding down an additionalbutton while annotating. For example, the user may press a button whileannotating if the user wants the annotation to continue in the plane ofthe first surface annotated. Any suitable control may be used to turnthe annotation method of FIG. 13 on or off.

In the present embodiment, virtual light is projected onto the sceneusing a virtual laser projector. In other embodiments, any suitablevirtual annotation tool may be used to project any virtual substance.For example, a virtual pencil, virtual pen or virtual spray can may beused to project virtual ink. The virtual substance may be projected froma distance, or may be projected from a position close to the surface ofinterest.

In other embodiments, the method of FIG. 13 may be used for any suitableapplication. For example, the method of FIG. 13 may be used in a designapplication, or in an educational or training application in which auser is being trained to perform a manual action. The method of FIG. 13may be used in a drawing or painting art application.

FIG. 16 is a flow chart illustrating in overview a method of anembodiment. The method of FIG. 16 may provide a combination ofreal-world physical movement and teleport, to allow actions such aswalking down steps and looking over ledges.

At stage 130, the processor 32 generates a computer-generated scenerepresentative of at least part of a house. In other embodiments, thecomputer-generated scene may be representative of any appropriatestructure.

The processor 32 displays the scene as viewed from a first viewingposition. The processor 32 displays the scene as if viewed from aviewpoint defined with respect to a virtual head. For example, theprocessor 32 may display the scene as if viewed from an appropriateheight above the ground to represent the height of a user, which may bethe height of a particular user who is viewing the scene.

At stage 132, the user provides a movement command to the processor 32by making a real-world physical movement. In the present embodiment, theuser walks and a change in position of the user's head is recorded. Themovement command is indicative of a movement from the first viewingposition to a second viewing position. Each of the first and secondviewing position may be a position of the virtual head of the user.Alternatively, each of the first and second viewing positions may bedefined as a position on a plan of the house, without an associatedheight, or may be defined in any suitable way.

In other embodiments, the movement command may be provided by anyappropriate physical movement of the user. For example, the user maymove their head, hand, foot, torso or other body part, and the virtualreality system 10 may translate the movement of the user into a movementcommand. For example, the user may step forward, and the virtual realitysystem 10 may translate the user's step into a movement command to moveforward. In some embodiments, the movement command may be provided bytracking of the user's physical body. For example, the user may wear abody suit. Tracking devices may be attached to feet, knees or other bodyparts. The movement command may be provided by way of a camera andsoftware capable of identifying body features.

In other embodiments, any suitable method of inputting a movementcommand may be used. The user may provide a movement command using aninput device, for example a handheld input device. The movement commandmay be representation of a continuous movement, or of a discontinuousmovement (for example, a teleport).

At stage 136, the processor 32 determines a movement of a virtual bodythat corresponds to the real-world physical movement of the user. In thepresent embodiment, the virtual body is considered to be placedvertically beneath a virtual head. The virtual body may be considered tobe dragged by the virtual head, in that the movements of the virtualbody are dependent on the movements of the virtual head. The virtualbody does not make movements independently of the virtual head.

In the present embodiment, the virtual body is represented by a capsule,which is a cylinder capped with two hemispheres. The virtual body may beconsidered to be a simple non-articulated body. Although in the presentembodiment the virtual body is placed directly beneath the head, inother embodiments the virtual body may be placed beneath and average ofthe virtual head and one or two virtual hands. In further embodiments,any representation of the virtual body may be used. For example, a morecomplex articulated body may be used. The more complex articulated bodymay in some circumstances allow for more accurate constraints andbehaviours than a simpler non-articulated body.

The virtual body may be coupled to the virtual head to maintain arelative position and orientation of the virtual body and virtual head,or the virtual body may be coupled so that it is moveable (for example,rotatable) relative to the virtual head. In some embodiments, a part ofthe virtual body may be moveable relative to another part of the virtualbody, for example to simulate bending at the waist.

The processor 32 determines the movement of the virtual body bysimulating the determined movement of the virtual head and consideringthe virtual body to be attached to, and hanging below, the virtual head.The processor 32 may simulate an effect of gravity on the virtual bodyand/or simulate an interaction of the virtual body with objects in thecomputer-generated scene.

In the present embodiment, if the virtual head moves horizontally in adirection in which a floor level changes vertically, the processor 32simulates gravity acting on the virtual body, causing the virtual bodyto move downwards if the floor level becomes lower. In the presentembodiment, if a downwards floor level change is small, the virtual bodyacts as if stepping downwards. If a downwards floor level change islarger, the virtual body acts as if falling.

In some embodiments, the virtual head moves in a direction in which afloor level changes upwards. If the upwards floor level change is small,the virtual body acts as if stepping upwards. If the upwards floor levelchange is large, the virtual body is blocked by the upwards floor levelchange. The virtual body may be deflected or stopped.

In some circumstances, the virtual head moves horizontally in a routethat would be navigable by the virtual head alone, but is not navigableby the virtual body because an object is present at a level of at leastpart of the virtual body. The processor 32 simulates the interaction ofthe virtual body with the object.

For example, the virtual head movement may comprise movement through aroom in which a table is placed. The table may not block passage of thevirtual head alone, but may block passage of the virtual body coupled tothe virtual head.

At stage 138, the processor 32 adjusts the movement of the virtual headthat was determined at stage 134 so as to take into account the movementof the virtual body that was determined at stage 136. For example, ifthe virtual body could not complete a movement from the first viewingposition to the second viewing position, the movement of the virtualhead may be curtailed so that it also does not move as far as the secondviewing position.

The processor 32 determines an adjusted second viewing position, whichmay be different from the second viewing position requested by the userusing the movement command. The second viewing position is an end pointof the adjusted movement of the virtual head.

At stage 140, the processor 32 displays the computer-generated scene asviewed from the adjusted second viewing position.

In some embodiments, the processor 32 gives the appearance ofteleporting by moving the virtual head from the first viewing positionto the adjusted second viewing position without showing any intermediatepositions. However, the position to which the virtual head is teleportedmay be affected by the determined movements of the virtual head andvirtual body.

In other embodiments, the processor 32 displays the computer-generatedscene such that the user appears to move through the computer-generatedscene from the first viewing position to the adjusted second viewingposition, with the user's viewing position moving in accordance with theadjusted movement of the virtual head.

In some embodiments, the user provides a succession of movement commands(for example, a succession of steps) and the processor 32 performs theprocess of FIG. 16 for each of the succession of movement commands.

In some embodiments, a position of the virtual body, or of part of thevirtual body is only determined by a position of the head and not by anorientation of the head. For example, if the user provides a command torotate their head, a direction from which the scene is viewed may bechanged, but there may be no change in a position of the virtual body.

The method described above with reference to FIG. 16 simulates aphysical body being dragged towards the position of the user's head, andthen obeying physical rules such as falling under gravity. In furtherembodiments, any suitable physical rules may be simulated. In somecircumstances, for example if the method of FIG. 16 is used in a game,the simulated virtual body may give or receive damage. For example, thevirtual body may be damaged if it walks through a fire. A more complexvirtual body may simulate feet, and may result in audible and/or visiblefootsteps. Any method may be used in which the virtual body is simulatedbased on the user's physical body movement.

In some embodiments, the movement of the virtual body is determined by amethod other than dragging the virtual body in dependence on a movementof the virtual head. The movement of the virtual body may be independence on a movement of any appropriate body part of the user. Themovement of the virtual body is adjusted in dependence on gravity and/orinteraction with one or more objects in the scene.

FIGS. 17 to 19 show an example of the method of FIG. 16 in which a userstands on a table and moves forward. The simulated body is detected ashaving moved off the table and hence the user falls to the floor. FIG.17 is an illustration of a screen shot of a computer-generated scene.FIG. 17 shows a first position 130 on top of a table 134. The firstposition may be a projection of a first viewing position (first virtualhead position) onto the table top. A direction of view is represented byarrow 132. FIG. 17 also shows an adjusted second position 136 (which maybe a projection of an adjusted second viewing position) and a directionof fall shown by arrow 138.

FIG. 18 shows a screen shot of the computer-generated scene as if viewedby a user standing on table 134 at first viewing position 130. The useris facing in the direction shown by arrow 132 and is facing downwards.

The user inputs a movement command requesting to move forward to asecond viewing position. The processor 32 determines a forward movementof the virtual head. The processor 32 then determines a correspondingmovement of the virtual body. When the virtual body moves forward inaccordance with the movement of the virtual head, gravity acts on thevirtual body and it falls downwards in the direction indicated by arrow138. The requested movement of the virtual head is adjusted to take intoaccount the falling of the virtual body. FIG. 19 shows a screen shot ofthe computer-generated scene viewed after the fall from the adjustedsecond viewing position 136.

FIGS. 20 to 22 show a user standing by a balcony and leaning forwards.The user provides a movement command indicative of leaning forwards. Inthe present embodiment, the user leans forwards in real life, and theprocessor 32 translates the user's leaning movement into a movementcommand which moves the virtual head from a first viewing position to asecond viewing position.

The processor 32 determines a movement of the virtual body in dependenceon the movement of the virtual head. The virtual body is detected ashaving collided with the balcony. The movement of the virtual body isconstrained by the balcony, and the movement of the virtual head isadjusted based on the constraint to the virtual body. Hence the userdoes not appear to fall in the virtual world, and can look over the edgeof the balcony.

FIG. 20 is a screen shot shows a general view of a computer-generatedscene including the balcony, which has a balcony railing 140. A firstviewing position is indicated in FIG. 20 by a circle 142, and adirection of viewing is indicated by arrow 144. FIG. 21 is a screen shotshowing the computer-generated scene as viewed from the first viewingposition, with the virtual head at an appropriate distance above thesurface of the balcony. FIG. 22 is a screen shot in which the user hasprovided a movement command indicative of leaning over the balcony. Thebalcony rail 140 constrains the virtual body, allowing a view of leaningover the balcony without falling.

In the embodiments described above with reference to FIGS. 16 to 22 , anentire virtual body is simulated. The virtual body is simulated as acapsule. The capsule comprises a cylinder capped with two hemispheres.In alternative embodiments, one or more parts of a virtual body may besimulated. For example, a torso, leg or arm may be simulated. In someembodiments, the virtual body has a shape resembling that of a humanbody. In other embodiments, the virtual body may have a simplifiedshape, for example a simple geometric shape such as a cylinder orcuboid. The virtual body may comprise a plurality of virtual body parts.The virtual body parts may be moveable relative to each other.

In some virtual reality simulations, for example virtual reality games,a user's head and/or hands are tracked with no physical simulation of abody of the user. In such virtual reality simulations, moving in thephysical world such that the user is suspended in mid-air in the virtualworld may not cause the user to fall in the virtual world.

In some virtual reality simulations, for example virtual reality games,the user may directly control a simulated body by controller inputs. Forexample, the user may instruct the virtual body to move forward orbackward, or to rotate. In such virtual reality simulations, physicalhead movements may be ignored or may not contribute to a position of thesimulated body.

By providing a method in which a virtual body is moved in dependence ona movement of a virtual head, a physical interaction of a body with itssurroundings may be simulated. Physical movements of the head maycontribute to what is seen by the user. However, unrealistic movementsof the head (for example, movements that cause the user to appear to besuspended in mid-air) may be avoided.

In other embodiments, the method of FIG. 16 may be used for any suitablevirtual reality application, for example in gaming. The method of FIG.16 may be used for virtual maps or virtual tourism.

FIG. 23 is a flow chart illustrating in overview a method of anembodiment in which a user may interact with virtual objects, forexample light switches or door handles, at a distance, rather thanneeding to be in close proximity with the virtual object in the virtualworld.

At stage 150 of FIG. 23 , the processor 32 displays a computer-generatedscene. In the present embodiment, the computer-generated scene isrepresentative of a room in a house. In other embodiments, thecomputer-generated scene may be representative of any suitablestructure. The computer-generated scene comprises one or moreinteractive virtual objects.

FIG. 24 shows a computer-generated scene comprising two interactivevirtual objects. The two interactive virtual objects are light switch160 and slider 162. The computer-generated scene is viewed from aviewing position such that the light switch 160 and slider 162 appear tobe at a distance from the user.

In other embodiments, the interactive virtual object may berepresentative of any real-world object that is capable of beingmanipulated by the user. For example, the interactive virtual object maybe representative of a real object that is configured to controllighting, sound, heat, air conditioning or another environmentalcontrol. The interactive virtual object may be representative of aswitch, button, knob, slider or any other suitable control.

In further embodiments, the interactive virtual object may notcorrespond to a real-life control. In some embodiments, an interactivevirtual object may be used to control a parameter that is not changeablein real life, for example, as a wall position, window position or floorlevel. In some embodiments, the interactive virtual object is an item,for example an item of furniture, that is configured such that it may bemoved by a user to a different position within the scene.

FIG. 24 also shows a virtual handheld input device 164 representative ofa real-life handheld input device 40 that is held by the user. In thedescription below, the user is described as operating the handheld inputdevice 164. Such operating results from a corresponding operation of thereal-life handheld input device 40. In other embodiments, a differentreal-life input device may be used to operate the handheld input device164. In further embodiments, a different input device may be representedin the scene.

The handheld input device 40 may also be referred to as a controller. Acontroller position and direction may be seen in FIG. 24 as the positionand pointing direction of virtual handheld input device 164.

At stage 152, the user points at an interactive object using thehandheld input device 40 to select the interactive object. A cursor maybe displayed to indicate a position at which the handheld input device40 is pointing. FIG. 25 shows the virtual handheld input device 164pointed towards the light switch 160.

At stage 154, the processor 32 displays a virtual manipulation indicatoroverlaid with or adjacent to the virtual interactive object that hasbeen selected by the user. The virtual manipulation indicator indicatesthat the virtual interactive object is in an active mode.

In FIG. 25 , the virtual manipulation indicator displayed by processor32 is a virtual hand 166 which is partially overlaid with the lightswitch 160. The virtual hand 166 is displayed as if poised to operatethe switch. A finger of the virtual hand 166 points to a toggle of theswitch. In the present embodiment, the virtual hand 166 is blue andsemi-transparent. The interactive virtual object that is currently beinginteracted with is indicated by the blue semi-transparent hand. Thevirtual hand 166 appears not to be connected to a body. For example, thevirtual hand 166 does not appear to be connected to a body of the user.

In other embodiments, any virtual manipulation indicator may be used.For example, the virtual manipulation indicator may comprise a region ofcolour superimposed on the interactive virtual object, a shapesuperimposed on or surrounding the interactive virtual object, or anarrow pointing at the interactive virtual object.

At stage 156, the user operates the handheld input device 164 tomanipulate the virtual interactive object. The operating of the handheldinput device 164 provides a manipulation command to the processor 32,which causes the processor 32 to manipulate the virtual interactiveobject. In the present embodiment, the user provides the manipulationcommand by pressing a button on the real-life handheld input device. Theprocessor 32 adjusts the computer-generated scene to simulate the effectof the light switch 160 switching, for example by turning on or off alight. In other embodiments, any method of providing the manipulationcommand may be used.

In response to the manipulation command, the processor 32 causes thevirtual hand 166 to move and the light switch 160 to switch.

By allowing operation of an interactive virtual object that isrepresentative of a real object (for example, a light switch) the usermay understand which controls would be available in a structurerepresented in the computer-generated scene, and what those controls do.

Stages 152 to 156 may be repeated for other virtual interactive objectsin the computer-generated scene.

For example, in FIG. 26 , the user has pointed the handheld input device164 at the slider 162. In response to the pointing of the handheld inputdevice 164 at the slider 162, the processor 32 displays the virtual hand166 such that it is partially overlaid on the slider 162.

The virtual hand 166 indicates which one of the virtual interactiveobjects is in an active mode. In the embodiment shown in FIGS. 24 to 26, the processor 32 puts the light switch 160 into an active mode inresponse to the user pointing at light switch 160. Then when the userpoints at slider 162, the processor 32 puts the light switch 160 into apassive mode and puts the slider 162 into an active mode. When a virtualinteractive object is in active mode, it may be controlled by sending amanipulation command via the handheld input device 164.

In other embodiments, any suitable virtual interactive objects may bepresent in the scene, for example a virtual lighting control, a virtualdoor control, a virtual window control or a virtual heating control. Avirtual interactive object may be representative of a real object thatis configured to control a door, a window, a window covering or anyother moveable item in the house (or in any suitable structure).Operation of each control may be simulated in response to a manipulationcommand. Operation of virtual lighting control may cause a change in thelighting of the computer-generated scene, for example a change inlighting intensity, colour, or direction. Operation of a virtual doorcontrol may allow a door to be opened or closed. Operation of a virtualwindow control may allow a window to be opened or closed, or may allowanother change in the window, for example a change in tint. Operation ofa heating control may cause, for example, a fire to be turned on or off.

In some embodiments, a virtual interactive object is representative ofan item of furniture. When the virtual interactive object is in activemode, the virtual interactive object may be moved by sending amanipulation command via the handheld input device 164. This may allowthe user to pick up and move furniture, which may create a simplevirtual interior design feature.

The virtual hand 166 (or, in other embodiments, a different virtualmanipulation indicator) may mimic a physical controller's movements. Forexample, the virtual hand may move around the scene in accordance with amovement of the handheld input device 164.

In the present embodiment, a virtual interactive object is selected bypointing at it using the handheld input device 164. In otherembodiments, any suitable method may be used to select the virtualinteractive object. For example, any suitable input device may be used.The virtual interactive object may be selected using a body movement ofthe user, for example a gesture of the user.

In the present embodiment, the manipulation command is provided bypressing a button on the handheld input device. In other embodiments,any method of providing the manipulation command may be used. In someembodiments, an action used to provide the manipulation command issimilar to an action that would be used to operate a real-life versionof the virtual interactive object (for example, switching a switch orsliding a slider). An action of the virtual manipulation indicator maymimic an action made by the user to produce the manipulation commandand/or an action that would be used to operate a real-life version ofthe virtual interactive object.

The method of FIG. 23 may combine proximity with direction to detectwhen an object is being pointed at. A virtual hand 166 is displayedinteracting closely with the virtual object, allowing the user tomanipulate objects from a distance.

Some virtual reality applications may allow interaction with virtualobjects using position-tracked controllers. Such virtual realityapplications may require the controller to make close virtual contactwith the virtual object before it can be manipulated.

By instead providing an interaction at a distance, the method of FIG. 23may provide a more convenient way for a user to operate virtual objectsin the computer-generated scene. For example, user may operate severaldifferent objects in different parts of a room from a single viewingposition. In some embodiments, the user may operate interactive virtualobjects that are more than 1 metre, 5 metres or 10 metres away in thevirtual world.

By displaying a virtual manipulation indicator (for example, a virtualhand), the user may know which object is currently active and thereforecapable of being manipulated by sending a manipulation command.

In other embodiments, the method of FIG. 23 may be used in any suitablevirtual reality application. The method of FIG. 23 may be used in manyvideo game VR contexts.

In embodiments described above, the methods of FIGS. 2, 5, 9, 13, 16 and23 are used to display an architectural structure to a user, for exampleto display a proposed buildings or proposed modifications to anarchitectural client, architect, planner, surveyor or builder. Infurther embodiments, the method of FIG. 2, 5, 9, 13, 16 or 23 may beused to view any suitable type of building. The building may be aresidential building, for example a house or apartment building. Thebuilding may be a commercial building, for example an office building,hotel, conference centre, or retail centre. The methods of FIG. 2, 5, 9,13, 16 or 23 may be used to view a streetscape or natural setting. Themethods of FIG. 2, 5, 9, 13, 16 or 23 may be used for any suitableapplication, for example in games, education or training, or historicalreconstruction.

In the embodiments described above, particular input methods have beendescribed, for example providing a command via a handheld input devicesuch as a games controller or wand. In other embodiments, any suitablemethod of input may be user. For example, a user may input a command viaany suitable input device, for example a keyboard, mouse, joystick ortrackball. Input may be provided by a user moving any part of theirbody, for example their head, hand, foot or torso. Input may be providedby touch or voice commands.

Features of the method of any one of FIG. 2, 5, 9, 13, 16 or 23 may becombined with features of the method of any other one or more of FIG. 2,5, 9, 13, 16 or 23 .

It will be understood that the present invention has been describedabove purely by way of example, and that modifications of detail can bemade within the scope of the invention.

Each feature disclosed in the description and (where appropriate) theclaims and drawings may be provided independently or in any appropriatecombination.

1. A method comprising: displaying in virtual reality acomputer-generated scene comprising an interactive virtual object, suchthat the interactive virtual object appears to be at a distance from auser; receiving from the user a remote selection of the interactivevirtual object; in response to the remote selection, displaying avirtual manipulation indicator overlaid with or adjacent to theinteractive virtual object, wherein the virtual manipulation indicatoris indicative of the interactive virtual object being in an active modein which the interactive virtual object can receive manipulationcommands; receiving a manipulation command from the user; and inresponse to the manipulation command from the user, operating or movingthe interactive virtual object, such that the interactive virtual objectappears to be operated or moved at a distance from the user.
 2. A methodaccording to claim 1, wherein the interactive virtual object comprisesat least one of a lighting control, a light switch, a door control, adoor handle, a sound control, a window control, a window coveringcontrol, a heating control, an air conditioning control.
 3. A methodaccording to claim 1, wherein the virtual manipulation indicatorcomprises a virtual hand.
 4. A method according to claim 1, wherein theoperating or moving of the interactive virtual object appears to beperformed by an action of the virtual manipulation indicator.
 5. Amethod according to claim 1, wherein the remote selection is provided bythe user by indicating the interactive virtual object using an inputdevice.
 6. A method according to claim 1 wherein the manipulationcommand is provided by the user using the or an input device.
 7. Amethod according to claim 1, wherein an action of the virtualmanipulation indicator mimics an action of the input device and/or anaction of the user.
 8. A method according to claim 1, wherein the remoteselection is provided by the user by pointing at the interactive virtualobject.
 9. A method according to claim 1, wherein an apparent distancebetween the user and the interactive virtual object is greater than 1metre, optionally greater than 5 metres, further optionally greater than10 metres.
 10. A method according to claim 1, wherein the or a structurerepresented in the computer-generated scene comprises at least one of anarchitectural structure, a building, a house, an apartment building, aresidential building, a commercial building, an office building.
 11. Acomputer program product comprising computer readable instructions thatare executable by a processor to perform a method according to claim 1.12. An apparatus comprising at least one display screen, at least oneuser input device, and at least one processor configured to: display invirtual reality on the at least one display screen a computer-generatedscene comprising an interactive virtual object, such that theinteractive virtual object appears to be at a distance from a user;receive from the user via the at least one user input device a remoteselection of the interactive virtual object; in response to the remoteselection, display a virtual manipulation indicator overlaid with oradjacent to the interactive virtual object, wherein the virtualmanipulation indicator is indicative of the interactive virtual objectbeing in an active mode in which the interactive virtual object canreceive manipulation commands; receive a manipulation command from theuser; and in response to the manipulation command from the user, operateor move the interactive virtual object, such that the interactivevirtual object appears to be operated or moved at a distance from theuser.